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ABSTRACT 

The relationship between mathematics tests and the 
theoretical learning process was explored using alternative 
statistical methods and models. Data for over 1300 students in grade 
5 using the mathematics subscales from the National Longitudinal 
Study of Mathematical Abilities (NLSMA) were analyzed. Results 
indicated that Bloom's taxonomy is weakly supported when a full model 
using both adjacent and non-adjacent paths (from path analysis) is 
examined. The factor analysis produced one factor, as Bloom's 
structure would dictate, but when the four factors accounting for the 
largest percentage of variance are examined, a factor structure was 
obtained which corresponded to only two factors that could be named, 
computation and problem solving. Analyses using the computer program 
LISREL IV, which accounts for large error components and sample 1 
sizes, supported Bloom's cumulative hierarchy. (MNS) 
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background Aim runit)sc or the study 



Mnth»»mtics testa yield direct measures of students* learning in 
mathematics. They, also, reflect unmeasurable cognitive procecree, 
that is intellectual abilities such as remembering facta, analyzing 
components, integrating concepts, making assumptions, evaluating 
statements or solutions. In^this study, the relationship between 
mathematics testa and the theoretical learning proceao is explored 
uaing sltemative statistical methods and models to mathematically 
describe rel*tionnhips between obnerred test scores and hypothetical 
constructs which we nssume contribute to the obseived tent scores. 
The teat results used ai a* results of the mathematics scales (ie. 
subtests) from th*» National Longitudinal Study of Mathematical 
Abilities (NLSHA, 196?-19^7) for grades 5, 8, 11, and 1?./ 

The test items were constructed following a sumulative hierarchic 
structure baaed on the first four categories in Bloom's Taxonomy of 
Educational Objectives for the Cognitive Domain (Bloom et .al, 195°) . 
Bloom's classification system builds from simple behaviors to c*«p3ex 
b*»havioia where simple behaviors are integrated with other simpi** 
l*»hnviors to foim a more complex one. The four clossificationn vur<{ 
here are: 



Classification 

1. Computation 
(Knowledge) 

? m Comprehension 

% Application 

Analysis 



Behavior 
A 

A * B 

A ♦ B ♦ C 

A * B ♦ C « P 



A represents the behavior that ia required to successfully answer 
a test-item requiring only Computation. That, in addition to another 
behavior, B, is required to succeed at a Comprehension-level item. 
Thun a highrr level in build from the previous level plus one addition 
»>obavior in n model that in additive, rather than, say multiplicative. 



Data that correopond to this cumulative structure have a correlation 
matrix that exhibits Outtman's <195'0 simplex structure where adjacent 
categories are more highly correlated than non-adjecent categories. 
Thus, a perfect simplex has a correlation matrix such that the largest 
values lie along the firot off-diagonal. The next largest values lie 
along the adjacent diagonals. The smallest values will than lie in the 
upper-right and lower-left corners of the matrix. 

A similar hierarchy is found in mathematics learning. Except for 
some basic definitions and assumptions, mathematios la built upon 
other prerequiaite mathematica regnrdleaa of the content area or 
Srade level. It seema natural, then, to believe that in order to 
learn a particular mathematical concept or fact t a student needs to 
know all of the concepts below that one in the hierarchy. Thus, 
lower levels of mathematics are the building blocks of higher levels 
of mathematics. It is this idea that is the genesis of a hierarchy of 
learning in mathematics. See Appendix A for a review of the literature. 

Previoua attempts to validate Bloom* a cumulative hierarchy of 
learning in several different subject areas have been inconclusive 
(Seddon, 1978). There is supportive evidence, however, particularly 
fr*« the simplex analysis of Kropp and Stoker (1966), the multiple 
regression analysis of Madaua, Woods, and Nuttall (l973), and most 
recently the reanalysis by Hill and McOaw (1981) of the Kropp and 
Stoker data uaing JoYeskog's LISREL (1978) program. 

Mathematica acalea consisting of groups of similar items from 
one test battery are classified into one of the above cognitive levels. 
In order to explore the validity of Bloom* s cumulative hierarchy 
path analyaia, factor analyaia, and the computer program LISREL IV 
are used to describe the relationships between these four cognitive 
levels. 

The path analysis model assumes there should be direct links between 
mathematics scales from adjacent ^cognitve levels and no direct links 
from non-adjacent levels. The possible paths between the four cognitive 
lc/els are illustrated below where adjacent paths are shown as solid 
arrows and non-adjacent paths as broken arrows* In this model, the 
rathematics test scores equal the underlying cognitive levels. 



^ Computation ^ ^ ^Comprehension j fr^ Application ^ fr ( Analysis j 

Figure 1. Path diagram i Hunt rating adjacent and non-adjacent patha. 

Path coefficiente are estimated in linear atructural equation 
modela derived from relations among the observed mathematics scales in 
terms of caune and effect variables and their indicators* Here, the 
scales are grouped into thooe scales classified at the same category for 
a particular grade level. 

Factor analysis investigates the theoreticel latent structure that 
could have produced the observed correlation matrix of the mathematica 
acalea. Here, it is the latent factor structure, in contrast to the 
relationships between observed test scorsa, that is manipulated. 
Aasuming we have four tests — one from each of the four cognitive 
levele under coneideraUon — a simplex structure in the correlation 
matrix o r these tests would be represented in the factor structure 
ahown below. 



~ Factors f\, *3 ^ 



0 o 
0 o 



— — — 0 



Figure ?. Factor structure of four tests whose correlation matrix 
exhibits a simplex structure. 

For this study, there ere usually several tests categorized at each 
cognitive level giving multiple indicators of eech level* Bloom's 
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stivcture implies that those tests that were classified under the sane 
cognitive level should exhibit the same factor structure. In this sense, 
certain mathematics tests go together. We then look for s factor structure 
that exemplifies the classifications of the tests and the cumulative 
ordering. 

In another factor analytic approach, the computer program LISREL IV 
integrates linear structural equation models involving observed test 
scores with latent variables corresponding to the four cognitive levels. 
Again* we investigate how well the theoretical simplex structure exists 
in the observed data. However, in this analysis, the relationships 
between the four cognitive levels from Bloom's hierarchy as latent 
variables and the observed mathematics test results are separated and 
made explicit. Furthermore, two models are specified and tested for 
their goodness of fit using a chi-equare statistic. The first model 
specifies the test classifications according to cognitive level but 
does not Unit the relationships between cognitive levels to Bloom's 
cumulative ordering* In this model there is complete symmetry between 
the four cognitive levels in that each one is connected to all of the 
others. Thir, first model is illustrated in Figure 3. 



Computation) 

t 

j ( Comprehension") } |y^j ^ . 

» f ApplicatloTT) } JyJJ« . e^ 

U Analysis) ' % 

y^ = observed tost i (i-1» ... , *») 
e^ = error in test i (1=1, ... , *0 

Figure 3. Hypothetical LISREL model ohowing all possible paths between 
the latent variables. 
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The first model is compared to a second model that differo from the 
first in that there is Bloom's hierarchical ordering in the four latent 
variables. 

\ 

\^ <i », ( Computation^ > {T^J < e i 

\ » ( Comprehension^ "* H3* * ? 

\ i 

\* 5 > (Application j > < « 3 

(Analysis) » «- % 

* observed test i (im1, ... , *») 
e\= error in test i ... i *) 

$ A *v error in latent variable i H= 1» i **) 

figure Hypothetical MSRKL model showing cumulntive hierorchy in 
tlic latent variables. \ 



By exploring the application or these threo various structural 
equation estimation technique to the NLSMA data, ench odds to our 
understanding or whether Bloom^ cumulative hierarchical ordering is 
exhibited in the data. The important question is not whether Bloom's 
hierarchy gives the only valid explanation or the test results, but 
rather in exploring the validity or Bloom's hierarchy how do these 
statistical irethods contribute to our understanding or the teat results. 



THE Nig HA DATA 



The National Longitudinal Study or Mathematical Abilities (NL5HA) 
began in 1962 to collect data on over 112,000 students rrom 1,500 
schools rrom '»C states in the United States. Date was collected tor 
5 years on students in grades Tour through twelve. This continues to 
be the largest such study or mathematical achievement in this country. 

Three portions or students were studied. The Mgttre below 
and the accompanying quotation rrom NLSHA describee the design or HLSHA. 
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I 



Or ad en 




X - Poptilotlon 



Y- Populotion 



Z - Populoilon 



the rtg»re above illustretss the design or HtSMA. A lergs population 
or studsnts at each of thre« giede lewis vas teste* In the feU*"* •prlng 
or etch year, beginning with grades 7, end 10 in the rail, 19W. the 
X- Population and T-Populetlon were tested ror rive yeare. the t- Population 
w tested ror thtee ycere snd then rollowed vith o/iestionnaires artsr grad- 
uating rro* high school . the design stressed three features r (l) the long- 
tent study or a group of ctudents - up to rive years, (21 »tudy or tto u«e 
grade level »t different tines - tor Instance, grides. 7-6 In 1968- w ror ths 
T- Population and again in 196v67 Tor the X-Populatioo, and (3) extensive 
data on anthem ties achievement ror grades % through }2. 



Figure 5« 
Reports.) 



The design of HLSMA. (Reprinted fro* FORE WORD to ell WLSHm 



Mathematical achievement was characterited in a matrix or three 
content areas that students typically covered in the rotirth through 
twelfth grade curricula and four cognitive level o«tegories taken froa 



s taxonomy. 


ffumber Systems 


Geometry 


Algebra 


Computation 








Comprehension 








Application 








Analysis 









Figure 6. HLSMA model for mathematics achievement. 
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Since the goal or the preoent atudy is to investigate the validity 
of the im^ned taxonomic structure in the mathematics scales, only 
those populstions thst include at least one scale Trom each cognitive 
level category ate considered. From the thirteen possible grades 
sprsnd over the HLSMA X-, I-, end Z-Populationa, Tour samples «ere round 
that met this criteria: X-Pt>pulation, Grade 5 and Orade 8j I-Populstion, 
Grade 11 (group Dj "id Z-Populstion, Orsd. 12 (grou'p 2). Group 1 in 
the grade 11 I-Population Includes all students who had completed at 
least three years or college preparatory msth.mstics by the end of that 
year. Approximately «»5 Percent or the T-Populstion, Grade 11 are in 
group 1. Group 2 in the grade 12 Z-Populetion includes all students -ho 
had not had st lesst one msthematics course more advanced than geometry. 
Approximately 30 percent or the Z-Population, Orsd. 12 are in group 2. 

In general, the atudents uere above average in mental sbility, 
mathematics achievment, and socio-economic status. They csme Trom 
schools rro. all «»• geographic region- in the united States - North 
Atlantic, Southeast, nidueot, Great Plaina and Bocky Mountains, and 
Far West. Statistical inrormation is presonted ror a 5 percent 
stratiried (by geography) random sample °f «ch or tha entire X-, I-, 
and Z-Populationa. This sampling procedure yields large sample oiv.es 
in the four snmpls. under investigation - 1776 -indents in the X-Popul.tion 
Grade 5| "30 students in ths X-Popul.tion Grade 8, 515 students in the 
group 1 I-Population Grade 11, and 205 atud.nt. in the group 2 Z-Popul.tion 
Grade 12. 

In this paper, we vill exa-ine the results Tor the grad* 5 population 
only. Descriptive statistics Tor this sample aro given in Table 1 and 
TsMe 2. Complete-test batteries and item statistics Tor each scsle 
are round in HLSMA Reports Moe. 1 - 6 (Wilaon, Cah.n, Begle, .1968). 
Correlation matrices on all or the HLSMA scales are round in HLSMA 
Report Ho. JJ (Wilson, Begle, 1972). 
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TAWtf 1 

X-IVipulQtloti Mathematics Scales for Spring l<»6H, Orade r >, 
:Vunple filrcn Rat^e frrom H20 - 133<>. 

*~ ~ " Number Tlme^ aii**«- Krr ° r ° f 

Cognitive U-vel toth Hcale of Itenle Allowed P Measurement 



Computation 


X301 - 


Froctionn 3 


10 


12 mln. 


O.Of 


1.11 




X302 - 


h»rlmals 2 


I 


V win. 


0.70 


1.03 




X3Q3 " 


Mvlclon Whole 
Numbers 2 


0 


15 mln. 


0.82 


l.o8 


Comprehension 


X30?» - 


Decimal Notation 


0 






1.2*1 




X305 - 


Trans Int Ion 


? 




o.Cn 


l .06 




X30o - 


(k'ouctric Piiancsi 


h 




0.53 


0.73 


Appl i cation 


X30/ - 


Working With 
Numbers 


12 


20 Bin. 


0.60 


i.h9 


Anulycls 


x^oO - 


Five Dote 


19 


IV mln. 




1.70 


• II time information l« not glvon, then the 
section. 


scale 


Is part of 


a turgor timed 


Alpha in nn 


eotJmnt e 


of the Internal cone latency 


reliability of the 


scale. 
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TAWJ? ? 

Means and Standard Deviations for X-Ibpulatlon Matticmatlcs Scales, 
Spring 1<>6H, Crade 5, N-1776. 



Cognitive Ijevel 




Mnth Scele 


Mean 


Stand urd 
Deviation 


TVrcenttl* 


Computation 


X301 


- Fractions 3 


5-97 


3.07 


0.60 




X302 


- Decimals 2 


3.20 


^.11 


O.h^ 




X303 


- Division Whole 
Numbers 2 


'».A3 


?.5'» 


0.60 


Comprehension 


X30H 


- Decimal dotation 


3.06 


1.03 


O.HO 




X305 


- Translation 


i».82 


1.87 


o.6l 




X30C> 


- Geometric figures 


2.57 


1.07 


0.6* 


Application 


X307 


- Working With 
numbers 


S.70 


2.59 


oM\ 


Analysis 


X308 


-Five Dots 


10.7H 


H.77 


0.25 



From Table 1 we" note that the small number ol itemo in esch ncnle 
yield leas reliable scale scores. Also, there are not equal numbers of 
itemo in each ocale. There Are not equal numbers of acales for each 
cognitive level category. In general, over the entire N1SHA population, 
there nre more scrfles at the lower cognitive level categories in the 
lower grndee and more ocaleo at the higher cognitive level categories 
in the higher grndea corresponding to the more sbotract quality of the 
materinl being taught at the higher grades. 

From Table 2 we note that the percentile menna generally decrease 

from lover to higher cognitive level category which i« consistent with 

bloom's taxonomic structure. 
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RESULTS FOR X-rOF»LAT10N. GRADE 3 

Results Uoing Path. Analysis * 

Using the KLSHA' class! fication of the m*the«atice scales, a pnth 
diagram ia uaed to display the pattern and strength of the caueal 
relations, between acale scores from adjacent cognitive levels. The 
friabilities of the indecent acalee at the lowest level, Compulation, 
are all ansumed determined by caueen outaide the model. The variabilitiee 
of the dependent ecalea at the other three levele are explained by the 
previous independent varinblee or by other dependent variable. 

Each path diagram representa a model that consists of a set of 
equations each of which includea a disturbance tert^that euKmarire* 
the effect on the atructnre of the system of both measurement error 
for that equation and nil other unknown variables. Each model io 
aosumod to represent linear relationohipe between the variablea and 
the diaturbance terms where the disturbancea are independent of each 
other and of nil variablea that precede them in the given caueal ordering* 




*6 



, 1= Fraetiono y 2 =Deciaale, y ^Division, of Whole Numbers, 
y^Decimal Notation, y ^Translation, y 6 =Geometric Figurea, 
y 7 =Working With Numbers, yg=Five Dots. 

Figure 7. Path dingram in a model using testa from Number Systems 
nnd Geometry, Grade % N=I776. 

I 
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Path coefficients, p AJ , where p^ represents the p»th from tent y^ 
to tost y $ are Riven in Figure 7 above. This diagram repreoents tho 
following five regrencion equations. 

y>, • 1*1,1 3-1 4 P»,2 y 2 4 P '.J y 3 4 e <. 
* 5 a ''51 h 4 p 5.= y 2 * p 53 y 3 + "5 
(1) h " p 6i y i 4 p 62 y 2 4 p 6j y 3 4 e 6 

r 7 ' 9jk ><* 4 p 75 7 5 4 P 76 '6 4 "7 
y 8 = 1B7 y 7 4 C 8 

Rplotionohlpo between the regreneed enrlablee y,, 1^ y ? may be preoented 
in a correlation matrix ns ahown In Table J. 

TAW.E 5 

Correlation Hatrlx For Computation Scalee, Orndo 5. 



Test: 


>i 


y 2 


y 3 


»1 


1,00 








0,<»9 


1.00 




y 3 


0,59 


o.<*5 


1.00 



Since the correlations nre computed on standardized variable*, the 
regrenaion coefficienta are identical to the path coefficienta obtained 
from solving a seriee of linear atructurar equatione. veing the model 
described in Figure 7, we obtain eetimatee for the path coefficients as 
nolutions of three nystemn of equations. From the equations 

>V> ' Phi * »V" *W ' ^3 r '3 
(?) '-A " P >U r l? ' P '«? * P, <3 r ?1 

&3I1 P M r 13 ' ^ ' r,, 3 
vc obtolr jetimateo of P b|( P,, 2 , P,^ *™n the equations . 
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(3) - P.,, V 1 P, r - P.* ^ 

ve obtain estimator of p ?J , p r>2 , P^: from tne equations 

we obtain estimates of p^, p 6? , p 6 y In each syatem of equations 
there are exactly thrno equations in three unknowns, and the quationft 
nre independent, oo that thene systems are identified, 

Ae a consequence of the last two regression equations of equation (1) 
we obtain the equations 



(5) 



r l/ • 


•v. 


r M 4 




r ',l 


' p rf- 


r 6l 


X 2i ' 


p f<. 


V 1 


p r, 




' V, 




r 37 " 


' 


•»3 


1 J 


.'J 


' p 76 




r hl ■ 






P I5 


M. 


* p 76 


r r,u 


Sf r 






P P, 




1 p 76 




r (-l ' 




v. ' 


p f. 




' V, 





Here we have o*x equations in three unknowns, so that the system is 
overidentified. Solving this system algebraically cannot Rive unique 
estimates. However, the regression procedure does give unique regres- 
sion coefficients which con be used os unbioaed estimates for the path 
coefficients. Similarly, the laat regression equation in (1) correspondo 
to an overidentified syotem of seven equations in one unknown. 
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All but two of the mathematics scaleo from the Orade 5 oample were 
originally classified under the content area Number Systems. The other 
two ocales were classified under Geometry. By collapsing ocrose these 
two content aroao, and using ordinary least squares regression, we obtair 
the estimates given in Figur e 7. Estimates for the disturbance terms 
were computed using"Y^" H * or eacn regression. 

For this sample, ve note only one path coefficient (p^r0.05) that 
is close to zero. The mathematics scale y 6 =Geometric figures consists 
of four multiple-choice questions dealing with naming figures that 
reprenent a square, circle, rectangle, and triangle. Working With 
Fractions (y^ and Decimals (y 2 > may be taught using geometric figures 
(pie charts, segmented rectangles, etc.) whereas Division of Whole 
Numbers (y^) typically is not illustrated using geometric fibres which 
may explain the difference in the strengths of these relationships. 

None of the path coefficients is very large, which *• would expect 
given the large sample size. We note, however, that the strength of 
the path coefficients generally increase as we go from low to high 
cognitive levels. Also, the disturbance terms are large and are 
inversely proportional to the number of items in the mathematica scale. 
That is, the largest disturbance terms correspond to those scales with 
the fewest* numbers of items. 

Because collapsing tha mathematics scales across the^two content 
areas may not be consistent with Bloom's taxonomic structure, similar 
regressions were carried out using only the scales from Num^r SyeUwu 
The path coefficients obtained are essentially the same. 

Regressions were also completed on this sample for the ill model, 
that is, including all of the non-ad Jacent paths. The non-adjacent path 
were all smaller than the adjacent paths; however they were not all tero 

The large error terms in rigure 7, compared to the values of the 
path coefficients themselves, preclude any strong interpretation of 
the results of this analysis. Thus we turn to factor analytic technique 
to aid us in our understanding of these results. 
- — 

Results Using Factor Analysis . 

Principal components analysis was used ss an exploratory technique 
to justify using models relating the four underlying hypothetical 
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cognitive levels of Bloom's Taxonomy as unobserved Intent variables to 
the obi>erved mathematics scale scores. First, corresponding eigenvalues 
greater than one vers extracted/ This procedure produced one principal 
component that accounted for approximately 3% of the total variance. 
Because the mathematics scales were categorise* into one or Tour 
cognitive levels, the first four principal components were also examined 
and rotated hoping to rind a structure consistent with the four 
hypothesized cognitive levels. The first four principal components 
accounted for approximately 81* of the .total variance. 

Bloom's cumulative hierarchy implies that one principal component 
should be sufficient to account for a large percentage or the total 
variance. Principal components analysis is also used to check the 
structure or each group or teats in a given cognitive level to determine _ 
in any particular scale loads very dUrerently than all the others in 
that category. (In that case, the results would indicate a misclassifi- 
cation or that scale.) For this sample, all or the tests at a particular 
cognitive level had similnr ractor loadings. 

Since principal components are orthogonal, and we expect an oblique 
ractor structure, we do not expect to rind a structure exemplirying 
Blcom»o hierarchical structure using principal components. We do, 
however, want to account for as much or the total variance as poosiblo. 
Therefore, principal components is used to determine the values or the 
rirst Tour eigenvalue* in order to specify Tour rectors in a claosical 
ractor analysis that will allow oblique rectors. 

Both orthogonal and oblique rotations are examined. The potentially 
low reliabilities of the ocalea limit the strength of the expected 
structure. The only interpretable structure that was round wao to 
secrete tho tests into those at the Computation level and those at 
aH the higher levels together, except the one geometry scale, Geometric 
Figures, produced its own unfque ractor. Because not all possible 
oblique ractor structures were examined, this analysis does not give 
a definitive answer to whether or not Bloom's hierarchy is exhited in 
this data. It does indicate, though, that classifying mathematics teata 
into those tapping two factors (««7 , computation and problem-solving) 
is warranted. 
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Results Uolnfi LISREL IV . 

Haximum likelihood rsctor analytic proceduren sre used in order to 
obtain initial values lor the parameters required in the LISREl* IV 
computer program. (We note that a new version or the program, LISREL V, 
produces its own initial values given one initial estimate.) 

This computer program provides the analysis tor a structural ^ 
equation model that is used to speciry explicitely the behavior or 
the NLSHA mathematics scales related to a cumulative hierarchical 
ordering of four cognitive levels. Joreskog introduced a very general 
model in 1973. The program description of the specUicatii n, estimation, 
and testing or the model with illustrations from social science research 
is given in Joreskog (1977). 

Tha eetimaten obtained are based on the method or maximum likelihood. 
Models are examined containing directly observed variables or unobserved 
hypothetical Construct variables. Lataot variables are assumed to be 
related to other observed variables end to each other. 

The general model for which USREt IV was designed is described by 
the following matrix equations x 

(6) fii' * 4 

(7) 7 - AjX 4 * 

(8) ' * «JV X £ 4 i 

whero fi^V are coerricient matrices; #J , £ represent independent and 
dependant latent variables, respectively; $ is a matrix of error terms; 
x, y represent independent and dependent observed variables, respectively, 
with corresponding matrices or error terms & and 4 • 

contain regression coefricienta or y on ^ and or x on £ , respectively. 
An estimated covariance matrix is computed whose elements are functions 
of eight parameter matrices — _^ y t ^1 |J» P» •nd the covariance 
matrices of 

i The program allows for both errors in the linear equations, including 
specification errors and disturbance terms, and errors in the observed 
variableo, including measurement errors and observational errors. 
The program yields eatimatea of tha residual covariance matrix and 
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and measurement error covariance matrix as wall mm estimates of the 
causal effects fro« the structural equations, provided sll the parameters 
in the model are identified. 

We use a special case of the structural aquation model where the 
observed mathematics scales are represented by dependent variables 
and our hypothesised cognitive levels are represented by four latent 
independent variables. The structural equation model (6) then reduces to 



(7) 



fix- 1 



and the only vector of observed variables ia y. 

The methodological goal is to reproduce a covarlaitce matrix whose 
elements are funcions of the four parameter matrlcea: £*.Ay 
and the covariance matrices of £ , and £ . Our special case ia 
identical to a factor analysis with the following differences. 
Therm is no restriction that there be fewer factors than variables or 
that the covariance matrix of the residuals Ve diagonal. The only 
requirement .is that the covariance matrix of dependent observed 
variables be nonsingulsr and that the model be identified. 

In the identification of the parameters, the sssumption is made 
that the distributions of the observed variablea are deacribed by their 
momenta of first and second order. That is, the information in moments 
of higher order is ignored. This assumption is valid if the distributions 
are multivariate normal. 

In the estimation and testing of the model, it ia assumed that the 
distributions of the observed variables are described by a mean vector 
and covariance matrix. The problem ia to fit the covariance matrix 
imposed by the model to the sample covariance matrix. In the process, 
aaxlmum likelihood estimates emerge! such estimate, are sfficient for 
large sampls sires. A test of the model is made using the chi-squnre 
* statistic under the assumptions of multidimensional normality «nd large 
sample sizes. 

For th. K md. 5 ~-Pl«. » 1« of h °« - 11 tMO " 0-,la 

fit th. obaTT.d correlation -atrlx of th. HLSW «th.~tlc B acnl.a. 
Th. flr«t pod.1 Includ.a th. four cognltl*. l.ir.la ««t dca ~>t contain 
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the theoretical cumulative hierarchy. That is, all possible patha 
between the cognitive levolo are included. The second model is the 
some as the first but ia restricted to only include the cumulative 
hierarchy. In both models, one parameter from each mathematics scale 
at n given cognitive level is restricted to equal one in order to 
aasess the relative effects of each scale at that level. Chi-square 
statistics for both models are given. A large decrease in the chi- 
nquare statistic compared to a small change in the degreea of freedom 
indicates an improvement in the model. That io, there is supportive 
evidence that Bloom's cumulative hierarchical structure does exist 
in the empirioal data. 

Because of the stratified cluster sampling procedure used by the 

« 

NL5HA investigators, these chi-square atatiatice are known to be too 
lnrge. The critical issue is not the fit, however j the critical ieoue 
ia the difference in chi-square atntiatics. 

Figures 8 and 9 show the USVEL results for the Grade 5 sample. 
Hodel 1 is the model without the hierarchy and model 2 is the model with 
the hierarchy. Model 2 was unidentified without fixing the vslue of 
the disturbance term € g . In order to compare these two models, 
model 1 was re-run restricting this estimate to the value 0.23 that 
had been obtained when it was allowed to be free. This did not change 
any of the estimates fiom model 1 and allowed model 2 to be identified 
by using this some value for €g. The chi-square statistic dropped from 
}J 2 e 192.9<» to ^ 2 iy m indicating a better model with Bloom's 

cumulative hierarchy. 

Becauae of the large sample site, the probability value aaeociated 
wiih the model including Bloom 1 a structure is not ststisticelly significant 
even though the real dual differencea are small indicating s good practical 
fit. The sampllrg method also implies thst the probability value should 
also be higher giving sn even better fit. 
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CONCLUSIONS 

Analysis of the mane for the Grade 5 eampl* are United h*coUoe of 
Hie difference In the* numbers of items in each ncnle and because of * the 
low reliabilities. These low reliabilities due mostly to the cnoll 
number of items in each scale contribute to the large error terms in 
the path analysis of this data. However, even w*th the large error 
terms, Bloom's taxonomy is weakly supported in this analysis when a 
full model using both adjacent and non-adjacent paths is examined. 
The factor analysis produces one factor as Bloom's structure would 
dictate, but when the four factors accounting for the largest percentage 
of variance are examined, we get a factor structure corresponding to 
only two factors not four that could be named computation and prohlem- 
aolving. Neither orthogonal nor oblique rotations yielded the desired 
factor structure. However, because ex* particular oblique factor 
structure may be difficult to find, the factor analysis here is 
inconclusive* On the other hand, the analysis using the computer 
program LISRKL IV, that does account for large error components and 
large sample sizes, does support Bloom's cumulative hierarchy. (We note 
that in the other grade levela ( 8,11, and 12) examined but not reported 
but not 1 reported in this paper, the analyses using USREL all gave results 
supporting Bloom's hierarchy, whereas the path analyses and factor 
analyses were not conclusive.) 
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APPENDIX A 

Review of the Literature or > Hiararchlca oi learning 

In this section we rsview some of the studies that nave, in the 
lust two decades, tried to validate learning hierarchies. Two types of 
otudiee emerge.' Firnt, especially in mathematics and science education 
studies, there are those that use Oagne 'a model of constructing a 
network of links between higher-order tasks that depend on ths mastery 
of a set of lower-order taske. The validation techniques are based 
on analyzing dependencies along the links. The second type of etudy 
deals with validating Moom'e taxonomy. In theee studies various 
statistical techniques have been used to test whether a elmplsx structure 
exists in the data. There are indicatione that modifications to the 
original ordering and/or additional conetrocte are needed to adequately 
explain the results of che Bloom taxonomy-type teets. 

' Capie and Jones 097D etudy a series of ratios derived from a 
phi-correlation matrix to validate a Oagne model. They suggest that 
the construction of a logical sequence is not necessary when all 
behaviors considered relevant are measured and each behavior paired 
with all othere for analysis. 

White <1973, 197*b) diecusses three major Oagne investi- 

gations as well as several later studies following the Oagne model. 
He identifies several weaknesses in all the previous studies including 
the lack of a statistical test that takes into account errors of 
measurement, small sample sites, use of only one question per element, 
delays in testing, and imprecise specification of component elements. 
Ths later studies propose ways of correcting these weaknesses. They 
support the later postulate of Oagne <1#8) that generalized intellectual 
skills are learned hierarchically whereas verbalized knowledge ia not. 
The later studies also illustrate a statistical test developed by 
White and Clark < 1 973> for calculating the critical value of the 
maximum number that could occur in the crucial cell (numbtr of subjects 
who answer .correctly higher-order tasks but not lower-order tasks) 
for particular probabilities under the hypothesis that the connection 
between the two elements is hierarchical. 
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Kropp and Stoker (19^6) offer the first major validation study of 
Bloom's hierarchy. They develop taxonomy-type tests with equal numbers 
of items for each class. A comparison of means for each cists shows 
that higher means occur in the lower levels, in general, over four 
subject areas. Their simplex analyait gives some support for a cumula- 
tive hierarchical structure though they note problems in interpreting 
the category •Knowledge 1 and have some reveraals in the ordering of 
the categories •Synthesis' and •Evaluation. 1 %l 

Hadaus, Woods, and Hut tall <197j) reanalyze a subset of the Kropp 
and Stoker date with a causal model using multiple regression procedures. 
They also compute a general mental ability factor {"^-factor) using 
principal components analysis on one standardized test* They find 
«mny indirect links that would invalidate the assumed hierarchy. All 
but one of these disappear, however, upon adding the V'-factor. They 
also report a decline in the magnitude of direct links aa processes 
progress from simple to more complex behaviors. 

Seddon (1978) summarized many of the previoua Bloom validation 
otudiea in terms of educational and psychological issues. The results 
he studies are inconsistent. Re questions the correlational properties 
of composite scores used in many of these atudiea and suggests they 
could be avoided using factor analyaia or smallest space analysis on 
the correlation matrix of individual items. 

Hlller (1979) ««*e path analysis, stepwise regression, commonality 
analysis and factor analysis in reanalyzing the Kropp and Stoker data. 
All methods reject s simpls hierarchical structure. The factor analysis 
and commonality analysis suggest a two-factor model. The |*th analysis 
suggests a branching in the order of the levela though the ordering of 
Knowledge — Comprehenaion — Application — Analysis remains in ths 
original order. 

Rill and Mcflaw (19B1) use LISflEL to reanalyze s modified version 
of the Kropp and 3toker data. Their results support the simplex 
assumption when the category Knowledge la delated. 

These previous studies yield conflicting results. They all, 
however, reiterate a need for more research into validating learning 
hierarchies and improvement of the atatistical methodology needed to 
adequately interpret the results. 
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